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The findings from this review do not refiect the fuii body of research evidence on the effects of providing 
high schooi students with information about the coiiege appiication process and coiiege costs. 


What is this study about? 

The study examined the effects of providing low- 
income, high-achieving high school seniors with a 
multi-component intervention program including col- 
lege application guidance, information about the costs 
of college, and a fee waiver for college applications. 

Students were identified using data from the College 
Board and ACT. A random sample of students was 
selected from those who scored in the top decile of 
the SAT I or ACT and had estimated family incomes 
in the bottom third of the income distribution of 
families with students in the twelfth grade. This 
review focuses on findings from students who were 
randomly assigned to the multi-component interven- 
tion program (n - 3,000) or a no-treatment compari- 
son condition {n - 3,000).^ 

The application guidance component of the inter- 
vention included information about deadlines and 
requirements for college applications at nearby insti- 
tutions, at the state’s flagship institution, and at in- 
and out-of-state selective colleges. The application 
guidance component of the intervention also included 
tables that compared colleges’ graduation rates 
and provided tools for students to explore colleges’ 
curricula, instructional resources, and housing. The 
information about the costs of college component of 
the intervention provided students with information 
on the amount spent on instruction, the list price of 
attendance, and net costs of attendance for different 
colleges and universities. This net cost information 
was presented for hypothetical families with incomes 


Features of Providing Information About College 
Application Process and College Costs 


The authors of this study designed a multi- 
component intervention that provided low-income, 
high-achieving high schooi seniors with college 
application guidance, information about college 
costs, and a fee waiver for college applications. 
Specifically, the multi-component intervention 
included the following components: 

• an application guidance component that included 
mailed packets with tables listing graduation rates 
for nearby colleges, state flagship colleges, and 
selective colleges; 

• a net cost component that provided students with 
information about the actual cost of attending 
specific schools, including instructional spending 
and net costs for nearby, state flagship, and 
selective colleges; and 

• a fee waiver component that provided students 
with no-paperwork fee waivers that could be used 
at 171 different selective schools. 


of $20,000, $40,000, and $60,000, but emphasized 
that the student’s actual cost of attendance at a given 
school would be unknown unless he or she actu- 
ally applied. Finally, students received a waiver that 
allowed them to apply to 171 selective institutions 
without paying application fees. This multi-compo- 
nent intervention was administered by mailing these 
materials to the students’ homes, 4-1 4 months after 
they registered for a college entrance examination. 
The cost to implement the multi-component interven- 
tion was estimated at about $6 per student. 
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What did the study find? 

The authors reported, and the WWC confirmed, a 
statistically significant impact of providing students 
with college application guidance, net cost informa- 
tion, and fee waivers on postsecondary applica- 
tion submissions and postsecondary enrollment 
outcomes. The authors reported that the multi- 
component intervention increased the percentage 
of students who (a) applied to a selective institution 
(from 55% to 67%), (b) were admitted to a selec- 
tive institution (from 30% to 39%), and (c) enrolled 
in a selective institution (from 29% to 34%), relative 
to the comparison condition. Students in the multi- 
component intervention condition also completed 
more applications and were admitted to more col- 
leges than students in the comparison group. All of 
these differences were statistically significant. 


WWC Rating 


The research described in this 
report meets WWC evidence 
standards with reservations 

Strengths: This study is a randomized controlled 
trial. 

Notes: Although students were randomized to the 
intervention and comparison conditions, there was 
a high level of non-response on the surveys used 
to collect post-intervention outcome measures. The 
study demonstrated baseline equivalence of the 
analysis samples for the outcomes presented in this 
WWC report. Therefore, this evidence meets WWC 
standards with reservations. 
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Appendix A: Study details 

Hoxby, C., & Turner, S. (2013). Expanding college opportunities for high-achieving, low income 
students. Stanford, CA: Stanford Institute for Economic Poiicy Research. Retrieved from 
http://siepr.stanford.edu 


Setting 

The study was conducted with low-income, high-achieving high school seniors in the United 
States. Participants received the intervention materials via postal mail and reviewed the inter- 
vention materials on their own. 

Study sample 

A national sample of low-income, high-achieving high school seniors was targeted for this 
intervention and identified using College Board and ACT data, census data, and other sources. 
Low-income students were defined as those with an estimated family income in the bottom third 
of the income distribution for families with a student in the twelfth grade, based on the 2007-1 1 
American Community Survey. High-achieving students were defined as those who scored in the 
top decile of test-takers of the SAT 1 or ACT (1300 math plus verbal on the SAT 1, or 28 on the 
ACT). These students are typically geographically dispersed and so cannot easily be reached 
by usual methods of Informing students about college. To assess whether Information would 
change students’ behavior, the sample was randomized to the Expanding College Opportunities 
Comprehensive (ECO-C) intervention {n = 3,000), a no-treatment comparison group (n = 3,000), 
or one of four other interventions which are not included in this single study report (see Endnote 
3). The analytic sample for the ECO-C Intervention and comparison conditions included 1 ,835 
students who completed a survey the summer after they were expected to graduate from high 
school, and/or completed a survey the summer after which they were expected to have com- 
pleted one year of college. The demographic composition of the original assigned sample was 
not reported. In the analytic sample for the ECO-C intervention and comparison conditions, 45% 
of the students were female and 53% were underrepresented minorities. 

Intervention 

group 

Intervention condition students were high school seniors who had taken the SAT/ACT, who 
scored in the top deciles of the SAT/ACT, had an estimated family income in the bottom third 
of the income distribution for families with a student in the twelfth grade, and did not attend a 
“feeder” high school (feeder schools were those In which more than 30 students In each aca- 
demic cohort typically scored In the top decile on college assessment exams). Students were 
randomly assigned to the ECO-C Intervention program, a no-treatment comparison condition, 
or to one of four other interventions that provided only one intervention component (the four 
single component Interventions are not Included in this single study review; see Endnote 3). 
The ECO-C intervention included {^) Application Guidance, (2) Net Cost, and (3) Fee Waiver 
components Intended to help high school seniors learn about their options for attending col- 
lege and provide them with materials to help organize their multiple college applications. 
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Comparison 

group 


Outcomes and 
measurement 


Support for 
implementation 

Reason for 
review 


The net cost intervention component provided students with information about net costs for 
low- to -middie-income students. This information included list prices, instructionai spending, 
and net costs of the state flagship university, at ieast one other in-state public coilege, nearby 
coiieges, a seiective in-state private coiiege, one out-of-state private liberal arts college, and 
one out-of-state selective university. The materiais emphasized that the students’ actuai cost 
of attendance at a given school would be unknown unless they actually applied to that school. 

The fee waiver Intervention component provided students with no-paperwork fee waivers that 
allowed them to apply to up to 171 selective colleges. 

The comparison condition did not receive the intervention packet, and therefore received 
treatment as usual. Comparison condition students were high school seniors who had taken 
the SAT/ACT, who scored in the top deciles of the SAT/ACT, had an estimated family income in 
the bottom third of the income distribution for families with a student in twelfth grade, and did 
not attend a “feeder” high school (feeder schools were those In which more than 30 students 
in each academic cohort typically scored in the top decile on college assessment exams). 
These students received no intervention services, but may have received college application/ 
admission/enrollment information from other sources as part of usual practices. 

College application, admission, and enrollment outcomes were based on student survey 
responses collected during the summer after they were expected to graduate from high 
school, and the summer after which they were expected to have completed their first year of 
college. For a more detailed description of these outcome measures, see Appendix B. 

The intervention materials were delivered in a packet via postal mail to the homes of high 
school seniors. No training of high school students or their families was reported. No imple- 
mentation support was reported. 

This study was identified for review by the WWC because it received significant media attention. 
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Appendix B: Outcome measures for each domain 


Postsecondary applications 

Number of applications submitted 

This outcome is based on responses to a survey that asked students to report the total number of postsecond- 
ary applications they submitted. The survey was conducted during the summer after which students were 
expected to graduate from high school. For this analysis, the study authors used a count measure of the total 
number of applications submitted. 

Applied to a peer/selective institution 

This binary outcome is based on responses to a survey that asked students to report the names of postsecond- 
ary institutions to which they submitted applications. These data were collected during the summer after 
which students were expected to graduate from high school. For this analysis, the study authors measured 
whether students applied to any "peer" institutions (public, private, liberal arts, or other institution type), where 
peer institutions are defined as those institutions where median student scores fall within five percentiles of a 
student's own score. 

Number of colleges to which admitted 

This outcome is based on responses to a survey that asked students to report the total number of postsecond- 
ary colleges to which they were admitted. The survey was administered during the summer after which students 
were expected to have completed one year of college. For this analysis, the study authors used a count measure 
of the total number of colleges to which students were admitted. 

Admitted to a peer/selective institution 

This binary outcome is based on responses to a survey that asked students to report the names of postsecond- 
ary institutions to which they were admitted. The survey was administered during the summer after which 
students were expected to have completed one year of college. For this analysis, the study authors measured 
whether students were admitted to any “peer” institutions (public, private, liberal arts, or other institution type), 
where peer institutions are defined as those institutions where median student scores fall within five percentiles 
of a student’s own score. 

Postsecondary enrollment 

Enrolled in a peer/selective institution 

This binary outcome is based on responses to a survey that asked students to report the names of postsecond- 
ary institutions to which they enrolled. The survey was administered during the summer after which students 
were expected to have completed one year of college. For this analysis, the study authors measured whether 
students were enrolled in a “peer” institution (public, private, liberal arts, or other institution type), where 
peer institutions are defined as those institutions where median student scores fall within five percentiles of a 
student's own score. 


Table Notes: The study also provided results for whether students submitted at least five applications, applied to a peer public university, applied to a peer private university, 
applied to a peer liberal arts college, applied/admitted/enrolled to an institution that was peer within a range of percentile points, 4-year graduation rates of colleges to which 
applied/admitted/enrolled, instructional spending of colleges to which applied/admitted/enrolled, student related spending of colleges to which applied/admitted/enrolled, median 
SAT scores of colleges to which applied/admitted/enrolled, filed a FAFSA, used any application fee waivers, and number of application fee waivers used. These outcomes were 
not included in this report because they were not eligible outcomes as specified in the protocol and/or were overlapping with the more comprehensive application and enrollment 
outcomes included in this report. 
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Appendix C: Study findings for each domain 


Mean 

(standard deviation) WWC calculations 


Domain and 
outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Postsecondary applications 

Number of applications 
submitted 

2011-12 

cohort 

1,835 

students 

5,56 

(nr) 

4.67 

(3,60) 

0.89 

0.23 

-f9 

<0,01 

Applied to a peer/ 
selective institution 

2011-12 

cohort 

1,748 

students 

67% 

55% 

12% 

0.25 

-fIO 

<0,01 

Number of colleges to 
which admitted 

2011-12 

cohort 

1,835 

students 

2.31 

(nr) 

2,06 

(1.37) 

0.25 

0.17 

+7 

<0,01 

Admitted to a peer/ 
selective institution 

2011-12 

cohort 

1,738 

students 

39% 

30% 

9% 

0,19 

+8 

<0,01 

Domain average for postsecondary applications 




0.21 

+8 

Statistically 

significant 

Postsecondary enrollment 

Enrolled in a peer/ 
selective institution 

2011-12 

cohort 

1,687 

students 

34% 

29% 

5% 

0.12 

-h4 

<0.05 


Domain average for postsecondary enrollment 0.12 +4 Statistically 

significant 


Table Notes: For mean difference, effecf size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for aii students 
who are given the intervention (measured in standard deviations of the outcome measure). The effect sizes reported here were computed by the WWC (using f-statistics from 
regression modeis reported in the originai study) and therefore differ from the effect sizes reported in the originai study, which estimated effect sizes using the standard devia- 
tion in the comparison group (rather than the pooied standard deviation, used in the WWC caicuiations). The improvement index is an aiternate presentation of the effect size, 
rejecting the change in an average student’s percentiie rank that can be expected if the student is given the intervention. The WWC-computed average effect size is a simpie 
average rounded to two decimai pieces; the average improvement index is caicuiated from the average effect size. The statisticai significance of the study’s domain average was 
determined by the WWC; the study is characterized as having a statisticaiiy significant positive effect because univariate statisticai tests are reported for each outcome measure, 
the effect for at ieast one measure within the domain is positive and statisticaiiy significant, and no effects are negative and statisticaiiy significant, accounting for muitipie com- 
parisons. nr=not reported. 

Study Notes: A correction for muitipie comparisons was needed and resuited in a WWC-computed p-vaiue of < 0.05 for the Postsecondary Applications domain; therefore, the 
WWC confirmed that the resuit in this domain was statisticaiiy significant. The p-vaiues presented here were reported in the originai study. 
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Endnotes 

■' Single study reviews examine evidence published in a study (supplemented, if necessary, by information obtained directly from the 
author[s]) to assess whether the study design meets WWC evidence standards. The review reports the WWC’s assessment of whether 
the study meets WWC evidence standards and summarizes the study findings following WWC conventions for reporting evidence on 
effectiveness. This study was reviewed using the Postsecondary Education topic area review protocol, version 2.0. A quick review of 
this study was released on April 17, 2013, and this report is the follow-up review that replaces that initial assessment. The WWC rating 
applies only to the results that were eligible under this topic area and met WWC standards with reservations, and not necessarily to all 
results presented in the study. 

^ There are no conflicts of interest to report. 

® Students were randomly assigned to the multi-component (ECO-C) intervention program (n = 3,000) or a no-treatment comparison 
condition {n = 3,000), or to one of four other single component interventions (n = 12,000). These single component interventions included 
an Application Guidance intervention {n = 3,000), a Net Cost intervention (n = 3,000), a Fee Waiver intervention (n = 3,000), and a Parent 
intervention {n = 3,000). This review only focuses on the reported effects of the multi-component ECO-C intervention program, relative to 
the comparison condition, given that this multi-component intervention included three of the four components that were in each of these 
four single component interventions (the Parent intervention was dropped for the multi-component intervention program). 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2014, March). 

WWC review of the report: Expanding college opportunities for high-achieving, low income students. 

Retrieved from http://whatworks.ed.gov 
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Glossary of Terms 

Attrition 


Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Improvement index 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Single-case design 
(SCD) 

Standard deviation 


Statistical significance 
Substantively important 


Attrition occurs when an outcome variable is not avaiiabie for aii participants initiaiiy assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review if it falls within the scope of the review protocol and uses either 
an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

Along a percentile distribution of students, the improvement index represents the gain 
or loss of the average student due to the intervention. As the average student starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned 
to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into intervention and comparison groups. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 2.1) for additional details. 
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